Moving picture encoding method and moving picture decoding method

ABSTRACT

A moving picture coding apparatus ( 100 ) includes a coding unit ( 110 ) that generates a coded stream by coding inputted pictures and a film grain coding unit ( 130 ) that codes a representative pattern by selecting film grain components equivalent to at least one macro block and regarding the film grains as a representative pattern. On the other hand, a moving picture decoding apparatus ( 200 ) includes a decoding unit ( 210 ) that generates decoded picture data by decoding the coded stream and a film grain preparation unit ( 230 ) that prepares a superimposition pattern based on the representative pattern and additional information, and a synthesizing unit ( 240 ) that synthesizes the decoded picture data with the superimposition pattern.

TECHNICAL FIELD

The present invention relates to a moving picture coding method forcoding pictures constituting a moving picture on a block-by-block basisand a moving picture decoding method for decoding the coded stream.

BACKGROUND ART

Film moving picture is made of silver halide crystals that are dispersedin photographic emulsion of the film. The respective pictures that arestored in such a photographic film are generated by exposing anddeveloping the silver halide crystals. In the case of color movingpictures, silver is chemically removed after the development. However,the structure of a silver crystal remains as small grains of dye evenafter the development. Since, silver crystals are formed randomly inemulsion, the grains are formed randomly and dispersed in the respectivepictures. Such a recognizable grain structure is called film grain, andFIG. 1 shows an example of such a grain structure.

A viewer of a moving picture that is being reproduced cannot recognizeeach of the grains that has a size of about 0.002 mm down to about atenth of that size. However, the viewer can recognize groups of grainsand recognize that it is film grains.

As the resolution of the generated pictures is increased, therecognition level of the film grains also increases. Especially, in thecase of reproducing cinema or a high-definition moving picture, suchfilm grains are clearly recognizable. On the other hand, in the case ofusing a standard or smaller television as a display, such film grainsare not conspicuous so much.

By the way, applications where moving pictures are used are on theincrease, such applications ranging from TV phone communication, a TVconference, a Digital Versatile Disk (DVD), to digital television. Whensending and recording moving pictures, a large amount of data must besent via a conventional communication channel whose frequency band islimited, and such data must be stored in a conventional storage mediumwhose capacity is limited. Compressing digital data is essential forsending and storing digital data using a conventional channel and amedium.

In order to compress moving picture data, a number of moving picturecoding standards have developed. Examples are: (a) a series of H. 26× ofthe ITU-T (the International Telecommunication Standard); (b) a seriesof MPEGx of the ISO (the International Organization forStandardization)/IEC (the International Electrotechnical Commission).The latest and most-developed video coding standard is the one called H.264 or MPEG4 AVC.

The approach for coding that becomes a basis of those standards is mademainly using the following steps. First, pictures are segmented intoblocks so that the data of the respective pictures constituting a movingpicture can be compressed on a block-by-block basis, a block being agroup of pixels. Next, the respective blocks of moving picture data aretransformed from spatial areas to frequency areas, the obtainedtransform coefficients are quantized, and then entropy coding isperformed on the quantized transform coefficients so that spatialredundancy of the respective pictures can be reduced. Further, usingother picture blocks that are in a time relationship, the variation fromother pictures are coded. This coding is performed using motionestimation and compensation technique.

Among various moving picture compression technique, the so-called hybridcoding technique is known as the most effective coding technique. Thehybrid coding technique is a hybrid of time and spatial compressiontechnique and a statistical coding. Here, technique of motioncompensation, Discrete Cosine Transform (DCT), quantization of DCTcoefficients, variable length coding (VLC) is used. The motioncompensation is for determining a motion between current picture and acoded picture, estimating the current picture based on the determinedmotion, and generating a differential picture indicating the differencebetween the current picture and the prediction picture.

In the case of performing coding using a lowered bit rate in the motionpicture compression like this, film grains are removed, resulting incausing a problem that such film grains damage a unique video quality.Generally, high-frequency components of a picture are removed in orderto reduce data amount in moving picture compression. Since such filmgrains are high-definition components and high-frequency componentsignals, they are removed in compression processing.

Here is conceived a method for processing film grain informationseparately from the contents of pictures. In the method, the film graininformation is removed from a video sequence, the film grain informationis parameterized based on a predetermined film grain model, and suchstatistical film grain parameters are sent to be added to the codedmoving picture data. Such film grain parameters can be sent in a form ofSEI (Supplemental Enhancement Information) in the MPEG4 AVC. In the SEImode in the present moving picture coding standard, additionalinformation of a bit stream to be sent is included in order to provide adisplay capability in the coded pictures. At present, SEI informationprovides picture freeze, picture snapshot, video segmentation,progressive refinement and keying. An object of these options is toprovide functions in a decoder that has a support function and in a bitstream.

FIG. 2 is a block diagram indicating the outline of sending film grainparameters. The moving picture data inputted in a coding apparatus 700are sent to the film grain removing filter 701, and such film grains areremoved from the moving picture data. A coding unit 702 performs astandard moving picture coding process on the moving picture data fromwhich film grains have been removed. The coded moving picture data issent to the corresponding decoding apparatus 800 as a coded data stream.

Here, the film grain removing filter 701 is realized by a motioncompensated temporal median filter. Since the film grain has a randomstructure, it can be easily removed from a picture sequence using atemporal median filter unless it moves. In the case of a video sequenceincluding motion in the contents of pictures, a higher level approachmust be made. Therefore, the temporal median filter follows the motionfrom picture to picture, performs filtering on the contents of eachpicture, and removes film grains.

The moving picture data from which film grain information is extractedis coded according to one of existing moving picture coding standards.Film grain information is sent to the film grain parameterization unit703, and the film grains are parameterized according to a statisticalmodel.

It is possible to perform color conversion and/or pixel interpolation,using an original signal format, concurrently parameterizing these filmgrains. As a typical example, film grain is modelled in RGB color spacein order to approximate the color configuration in photographic film.Or, monochrome film grains can be added to Y components (brightnesscomponents) in YUV color space.

A simple method for parameterizing such film grains is to process suchfilm grains as Gaussian noises added to picture signals. In a film grainparameterization model with higher fidelity, there is a need to senddifferent parameters for each color component and/or for each grainlevel set.

The resulting film grain parameters are included in the moving picturein a form of SEI message and sent.

The coded video data and the SEI message including film grain parametersare sent to a decoding apparatus 800. The decoding apparatus 800includes a decoding unit 801 that decodes coded moving picture data anda film grain simulation unit 802. The film grain simulation unit 802generates film grains according to the received film grain parameters,and adds the generated film grains to the decoded moving picture data.In this way, it is possible to reproduce moving picture data on whichfilm grains are superimposed.

The film grain simulation is performed at completely decoding apparatus800 side. The film grain simulation depends on a predetermined model forreproducing film grains on a predetermined virgin film. Otherwise, thefilm grain simulation is performed by parameterizing a configurablemodel. Also, the film grain simulation is performed after the movingpicture data is decoded and before the moving picture data is displayed.

The drawback of this method is that film grain information must bestandardized according to a known statistical film grain model in orderto obtain respective parameters. Therefore, only film grains that aretrue of a standard statistical film grain model can be codedappropriately so that they can be correctly decoded and reproduced andthen sent.

A conventional film grain removing method like this requires high-levelcalculation and complicated hardware especially because of motioncompensation filtering.

Also, another moving picture coding method and moving picture decodingmethod has proposed (for example, refer to Japanese Laid-Open PatentApplication No. 8-79765). In the method, noise amount included inpictures are detected when coding moving pictures, a flag indicating thenoise amount is coded and transmitted, the bit stream is decoded, andthen white noises are added according to the flag.

However, in the moving picture coding method and the moving picturedecoding method, white noises are added uniformly according to the flagindicating the noise amount that have been removed using a pre-filter,and the methods entail a problem that the reproducibility of pictures ismade low.

Also, in the case of coding fine pictures that have parts where a lot ofhigh-frequency components are used, for example, parts of green trees,in a lowered bit rate, such high-frequency components are removedtogether with such film grains, and the methods entail a problem thatfine parts cannot be reproduced.

DISCLOSURE OF INVENTION

Therefore, the present invention is conceived considering theabove-mentioned situation. An object of the present invention is toprovide a moving picture coding method and a moving picture decodingmethod that can improve the reproducibility by recovering the partswhere a lot of film grains and high-frequency components are includedeven when using a low bit rate.

In order to achieve the above-mentioned object, the moving picturecoding method in the present invention is for coding each pictureconstituting a moving picture on a block-by-block basis and includes: afirst coding step of coding a current picture to be coded; an extractionstep of extracting high-definition components from the current pictureto be coded; and a selection step of selecting high-definitioncomponents equivalent to at least one block, from the high-definitioncomponents extracted in the extraction step.

Here, in a first aspect of the present invention, it is preferable thatthe moving picture coding method further includes a second coding stepfor coding the high-definition components equivalent to the at least oneblock, the high-definition components being selected in the selectionstep.

Also, the moving picture decoding method in the present invention is fordecoding a coded moving picture whose pictures that has been coded on ablock-by-block basis. It includes a first decoding step of decoding thecoded moving picture and generating decoded picture data, an obtainmentstep of obtaining high-definition components equivalent to at least oneblock, and a superimposition step of superimposing, on the decodedpicture data, the high-definition components equivalent to the at leastone block that has been obtained by the obtainment step.

Note that, here in the high-definition components, film grain componentsare included.

In this way, even in the case where inputted pictures are coded using alow bit rate, it is possible to improve the reproducibility byrecovering the high-definition components. Also, it is possible toimprove later the picture quality of the coded stream that is sent usinga low bit rate.

Further, the present invention can be realized not only as a movingpicture coding method and a moving picture decoding method like this,but also as a moving picture coding apparatus and a moving picturedecoding apparatus that are equipped with units corresponding to theunique steps included in the moving picture coding method and the movingpicture decoding method like this, and as a program causing a computerto execute these steps. Also, the program like this can be distributedthrough a recording medium such as a CD-ROM and a communication mediumsuch as the Internet.

As clear from the above-mentioned description, with the moving picturecoding method and a moving picture decoding method in the presentinvention, it becomes possible to improve the reproducibility byrecovering the high-definition components even in the case where theinputted pictures are coded in a low bit rate. Also, it becomes possibleto improve the picture quality of the coded stream that has been sent ina low bit rate later.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic diagram showing a structure of film grains.

FIG. 2 is a block diagram showing the outline of sending film grainparameters.

FIG. 3 is a block diagram showing the overall configuration of a movingpicture coding apparatus and a moving picture decoding apparatus inwhich the moving picture coding method and the moving picture decodingmethod in a first embodiment of the present invention are used.

FIG. 4 is a block diagram showing the structure of the coding unit inthe moving picture coding apparatus.

FIG. 5 is a block diagram showing the structure of the film grain codingunit of the moving picture coding apparatus.

FIG. 6A is a block diagram showing a first structural example of theselection unit in the film grain coding unit.

FIG. 6B and FIG. 6C are diagrams showing examples of histogramsrespectively.

FIG. 7 is a block diagram showing a second structural example of theselection unit in the film grain coding unit.

FIG. 8A is a block diagram showing a third structural example of theselection unit in the film grain coding unit.

FIG. 8B is a diagram showing an example of pixel values of eachsegmented macro blocks.

FIG. 8C is a diagram showing an example of film grain components for onemacro block that have been obtained from the calculation performed by amedian calculation unit.

FIG. 9A is a diagram showing an example of film grain components for onemacro block.

FIG. 9B is a diagram showing an example of components outputted by avariable length coding unit.

FIG. 10 is a flow chart showing the operation performed by the movingpicture coding apparatus.

FIG. 11 is a block diagram showing the structure of a decoding unit ofthe moving picture decoding apparatus.

FIG. 12 is a block diagram showing the first structural example of thefilm grain preparation unit.

FIG. 13A to FIG. 13H are diagrams showing examples of modificationpatterns respectively.

FIG. 14A to FIG. 14C are diagrams showing examples of modificationpatterns respectively.

FIG. 15 is a block diagram showing a second structural example of thefilm grain preparation unit.

FIG. 16 is a diagram showing the example where the synthesizing unitsynthesizes decoded picture data with superimposition patterns.

FIG. 17 is a flow chart showing the operation performed by the movingpicture decoding apparatus.

FIG. 18 is a block diagram showing the structures of the coding unit andthe film grain coding unit of the moving picture coding apparatus in asecond embodiment of the present invention.

FIG. 19 is a block diagram showing the structure of the decoding unit ofthe moving picture decoding apparatus in the second embodiment of thepresent invention.

FIG. 20 is a block diagram showing the first structural example of thefilm grain preparation unit.

FIG. 21 is a block diagram showing the second structural example of thefilm grain preparation unit.

FIG. 22 is a block diagram showing the overall configuration of themoving picture coding apparatus and the moving picture decodingapparatus where the moving picture coding method and the moving picturedecoding method in a third embodiment of the present invention are used.

FIG. 23 is a block diagram showing the structure of the moving picturedecoding apparatus in a fourth embodiment of the present invention.

FIG. 24A to FIG. 24C are illustrations concerning a recording medium forstoring a program causing a computer system to realize the movingpicture coding method and the moving picture decoding method in therespective embodiments.

FIG. 24A is an illustration showing an example of a physical format of aflexible disc that is a recording medium.

FIG. 24B is an illustration of the front view and the side view of thecase of the flexible disc and the front view of the flexible disc body.

FIG. 24C is an illustration showing the structure for recording andreproducing the above-described program on the flexible disc FD.

FIG. 25 is an illustration showing the overall configuration of thecontent supply system that realizes a content distribution service.

FIG. 26 is an illustration showing an example of a mobile phone.

FIG. 27 is a block diagram showing the internal structure of the mobilephone.

FIG. 28 is an illustration showing the overall configuration of a systemfor digital broadcasting.

BEST MODE FOR CARRYING OUT THE INVENTION

Embodiments of the present invention will be described with reference tofigures.

FIRST EMBODIMENT

FIG. 3 is a block diagram showing the overall configuration of themoving picture coding apparatus and the moving picture decodingapparatus that use the moving picture coding method and the movingpicture decoding method in a first embodiment of the present invention.

The moving picture coding apparatus 100 is for coding film grainsseparately from main pictures, and includes a coding unit 110 and a filmgrain coding unit 130 as shown in FIG. 3. On the other hand, the movingpicture decoding apparatus 200 is for superimposing such film grains ondecoded pictures, and includes a decoding unit 210, a film grainpreparation unit 230 and a synthesizing unit 240 as shown in FIG. 3.

First, the moving picture coding apparatus 100 will be described.

FIG. 4 is a block diagram showing the structure of the coding unit 110of the moving picture coding apparatus 100.

The coding unit 110 includes a control unit 111, a prediction residualcoding unit 112, a variable length coding unit 113, a predictionresidual decoding unit 114, a picture memory 115, a motion vectorestimation unit 116, a motion compensation coding unit 117, adifferential calculation unit 118, an addition calculation unit 119 anda switch 120.

The motion vector estimation unit 116 estimates a motion vector thatindicates an amount of motion to a position in an image area that isclosest to an inputted image in an area to be searched in the picture byusing the decoded image data after being coded as a reference picture.

The motion compensation coding unit 117 determines a coding mode of ablock using the motion vector estimated by the motion vector estimationunit 116, and generates prediction image data based on this coding mode.This coding mode indicates how a macro block is to be coded.

The differential calculation unit 118 calculates the differentialbetween the inputted input image data and the prediction image datainputted by the motion compensation coding unit 117, and generatesprediction residual image data.

The prediction residual coding unit 112 performs coding includingfrequency transform such as DCT (discrete cosine transform) andquantization on the inputted prediction residual image data, andgenerates coded data. The variable length coding unit 113 performsvariable length coding and the like on the inputted coded data, andfurther, generates a coded stream by adding the information on themotion vector and the information on the coding mode that are inputtedby the motion compensation coding unit 117.

The control unit 111 controls a parameter in the quantization performedby the prediction residual coding unit 112, the switch 120 and the like.

The prediction residual decoding unit 114 performs decoding processingincluding inverse quantization and inverse frequency transform on theinputted coded data so as to generate decoded differential image data.The addition calculation unit 119 adds the decoded differential imagedata inputted by the prediction residual decoding unit 114 to theprediction image data inputted by the motion compensation coding unit117 so as to generate the decoded image data. The picture memory 115stores the generated decoded image data.

FIG. 5 is a block diagram showing the structure of the film grain codingunit 130 of the moving picture coding apparatus 100.

The film grain coding unit 130 includes an extraction unit 131, aselection unit 132 and a variable length coding unit 133 as shown inFIG. 5.

The extraction unit 131 calculates the differential between the inputtedinput image data and the decoded image data inputted by the additioncalculation unit 119 so as to extract the film grain components.

The selection unit 132 segments the film grain components of the pictureto be coded extracted by the extraction unit 131 into a macro block of,for example, 16×16 pixels, and selects film grain components equivalentto at least one macro block and outputs it as a representative pattern.Three structural examples of this selection unit 132 will be describedbelow.

FIG. 6A is a block diagram showing a first structural example of theselection unit 132 of the film grain coding unit 130.

The selection unit 132 includes a block segmentation unit 1321, avariance calculation unit 1322, a histogram generation unit 1323 and ablock selection unit 1324.

The block segmentation unit 1321 segments the film grain components ofthe picture to be coded that has been extracted by the extraction unit131 into a macro block of, for example, 16×16 pixels.

The variance calculation unit 1322 calculates the variance for eachmacro block segmented by the block segmentation unit 1321, that is, thevariance of the pixel value of each pixel constituting each macro block.This variance can be calculated using, for example, the followingexpression.

Here, “N” is the number of pixels constituting a macro block, and “x” isa pixel value of each pixel.

The histogram generation unit 1323 generates a histogram of the variancecalculated by the variance calculation unit 1322 as shown in FIG. 6B,and notifies the block selection unit 1324 of which macro blocks havethe variances respectively indicated by the frequency peaks in thehistogram. Note that, since there are plural macro blocks, which havethe variances respectively indicated by the frequency peaks in thehistogram, plural macro blocks are notified to the block selection unit1324.

The block selection unit 1324 selects a macro block from among macroblocks notified by the histogram generation unit 1323, and the filmgrain components of the macro block are outputted as a representativepattern. Here, the macro block to be selected may be any macro blocksnotified by the block selection unit 1324.

Also, the block selection unit 1324 further specifies, as additionalinformation, a permitted pattern in superimposition or a prohibitedpattern in superimposition of the film grain components to be outputted.This permitted pattern or a prohibited pattern is specified for eachpicture or for each of the film grain components. This permitted patternor a prohibited pattern will be described later in the description ofthe structure of the moving picture decoding apparatus.

Note that the histogram generation unit 1323 notifies the blockselection unit 1324 of which macro blocks have two variancesrespectively, for example, in the case where there are two variancesindicated by frequency peaks respectively in the generated histogram asshown in FIG. 6C. After that, the block selection unit 1324 selects eachmacro block that has one of the variances, that is, two macro blocks intotal, and outputs the film grain components of the respective macroblocks as representative patterns.

FIG. 7 is a block diagram showing a second structural example of theselection unit 132 of the film grain coding unit 130.

The selection unit 132 includes a block segmentation unit 1321, avariance calculation unit 1322, and a block selection unit 1325. Notethat the block segmentation unit 1321 and the variance calculation unit1322 are the same as the ones shown in FIG. 6A.

The block selection unit 1325 holds a variance that has been previouslyset. Also, the block selection unit 1325 selects a macro block, fromamong macro blocks, which has the same variance as a predeterminedvariance among the variances calculated by the variance calculation unit1322, and outputs the film grain components of the macro block as arepresentative pattern.

Also, the block selection unit 1325 specifies, as additionalinformation, a permitted pattern in superimposition or a prohibitedpattern in superimposition of the film grain components to be outputted,likewise the block selection unit 1324 that is the above-described firststructural example.

FIG. 8A is a block diagram showing a third structural example of theselection unit 132 of the film grain coding unit 130.

The selection unit 132 includes a block segmentation unit 1321 and amedian calculation unit 1326. Note that the block segmentation unit 1321is the same as the one shown in FIG. 6A.

The median calculation unit 1326 calculates a median of pixels in therespectively corresponding pixel positions of macro blocks segmented bythe block segmentation unit 1321. In an example case of respective macroblocks segmented as shown in FIG. 8B (4×4 pixels in FIG. 8B), mediansare calculated in the following sequence: first, the median of “0”, “2”,. . . , and “−2” that are the pixel values at the left corners; next,the median of “3”, “2”, . . . , and “1” that are pixel values of theright neighbors of the pixel values at the left corners; and next, themedian of “1”, “3”, . . . , and “−1” that are the pixel values of thenext right neighbors, and the other medians are calculated. After that,the median calculation unit 1326 outputs the calculated median of eachpixel position as film grain components for one macro block. In otherwords, in the case of the above-described example, the film graincomponents that have pixel values of “0”, “2”, “1”, . . . , is outputtedas the representative pattern for one macro block, as shown in FIG. 8C.

Also, the median calculation unit 1326 specifies, as additionalinformation, a permitted pattern in superimposition or a prohibitedpattern in superimposition of the film grain components to be outputted,likewise the block selection unit 1324 that is the above-described firststructural example.

Note that the above-described first structural example may be configuredin a way that only the macro blocks whose variances are at apredetermined threshold or below among variances of the respective macroblocks calculated by the variance calculation unit 1322 are notified tothe histogram generation unit 1323.

The variable length coding unit 133 codes the representative patternsoutputted by the selection unit 132. For example, in the case where thepixel value of the image to be coded is represented in 8 bits, since therepresentative pattern outputted by the selection unit 132 is adifferential value, it can be represented in 9 bits. For example, in thecase where the film grain components for one macro block that is therepresentative pattern outputted by the selection unit 132 hascomponents shown in FIG. 9A, the variable length coding unit 133 outputsthe components shown in FIG. 9B in 9 bits as they are.

Note that the variable length coding unit 133 may perform frequencytransform such as DCT (discrete cosine transform) on the representativepattern outputted by the selection unit 132 and code the coefficients.In this case, it is desirable that no quantization be performed so thatthe representative pattern can be completely reproduced, butquantization where a small quantization parameter is used may beperformed.

Next, the operation of the moving picture coding apparatus structuredlike above will be described. Here will be described the case where theselection unit 132 of the film grain coding unit 130 is the firststructural example. FIG. 10 is a flow chart showing the operationperformed by the moving picture coding apparatus in this case.

The images to be inputted are inputted to the coding unit 110 and thefilm grain coding unit 130 on a picture-by-picture basis in order. Thecoding unit 110 codes the pictures to be coded and generates a codedstream (Step 101). Also, the coding unit 110 generates decoded imagedata and outputs the data to the film grain coding unit 130 (Step 102).The extraction unit 131 of the film grain coding unit 130 calculates thedifferential between the inputted image data and the decoded image datainputted by the addition calculation unit 119 of the coding unit 110,and extracts the film grain components (Step 103).

The block segmentation unit 1321 segments the film grain componentsextracted by the extraction unit 131 into a macro block of, for example,16×16 pixels (Step 104). Next, the variance calculation unit 1322calculates the variance for each macro block segmented by the blocksegmentation unit 1321 (Step 105). The histogram generation unit 1323generates a histogram of the variance calculated by the variancecalculation unit 1322, and notifies the block selection unit 1324 ofwhich macro block has the variance indicated by the frequency peak inthe histogram (Step 106).

The block selection unit 1324 selects one macro block from among macroblocks notified by the histogram generation unit 1323, and outputs thefilm grain components of the macro block as a representative pattern(Step 107). Also, the block selection unit 1324 specifies, as additionalinformation, a permitted pattern in super imposition or a prohibitedpattern in superimposition corresponding to the representative patternto be outputted (Step 108). Next, the variable length coding unit 133codes the representative pattern outputted by the selection unit 132 andthe additional information (Step 109). The coded representative patternand additional information are sent in another coded stream separatelyfrom the coded stream generated by the coding unit 110. Note that thecoded representative pattern and additional information can be sent asuser data of the coded stream generated by the coding unit 110.

Next, the moving picture decoding apparatus 200 will be described.

The moving picture decoding apparatus 200 includes a decoding unit 210,a film grain preparation unit 230 and a synthesizing unit 240 as shownin FIG. 3 as described earlier.

FIG. 11 is a block diagram showing the structure of the decoding unit210 of the moving picture decoding apparatus 200.

The decoding unit 210 includes a variable length decoding unit 211, aprediction residual decoding unit 212, a picture memory 213, a motioncompensation decoding unit 214, a switch 215 and addition calculationunit 216.

The variable length decoding unit 211 extracts, from the inputted codedstream, the information on a decoding mode and various data such as theinformation on the motion vector that has been used in coding. Theprediction residual decoding unit 212 decodes the inputted predictionresidual coded data, and generates prediction residual image data. Themotion compensation decoding unit 214 generates motion compensationimage data based on the information on the decoding mode and theinformation on the motion vector.

The addition calculation unit 216 adds the prediction residual imagedata inputted by the prediction residual decoding unit 212 to the motioncompensation image data inputted by the motion compensation decodingunit 214, and generates the decoded image data. The picture memory 213stores the generated decoded image data.

The film grain preparation unit 230 prepares a superimposition patternbased on the film grain components and the additional information sentby the moving picture coding apparatus 100. Two structural examples ofthis film grain preparation unit 230 will be described below.

FIG. 12 is a block diagram showing the first structural example of thefilm grain preparation unit 230.

The film grain preparation unit 230 includes a variable length decodingunit 231, a modification unit 232, a modification pattern selection unit233, and a random number generation unit 234.

The variable length decoding unit 231 performs variable length decodingon the representative pattern and the additional information that havebeen coded by the moving picture coding apparatus 100 and sent. Therandom number generation unit 234 generates a modification pattern as towhat modification the modification unit 232 makes, by generating arandom number using a random function. For example, the followingpatterns are determined: (a) as it is (no modification is made on it) asshown in FIG. 13A; (b) it is reversed as shown in FIG. 13B; (c) it isreversed and rotated to the right by 90 degrees as shown in FIG. 13C;(d) it is reversed and rotated to the right by 90 degrees as shown inFIG. 13D; (e) it is rotated by 180 degrees as shown in FIG. 13E; (f) itis reversed and rotated by 180 degrees as shown in FIG. 13F; (g) it isrotated to the left by 90 degrees as shown in FIG. 13G; and (h) it isreversed and rotated to the left by 90 degrees as shown in FIG. 13H.

Also, for example, the following pattern can be determined: therepresentative pattern of the area 50 shown in FIG. 14A is shifted tothe left as a modification pattern as shown in FIG. 14B, and the area 51that has left the area 50 is moved to the area 52 that has become vacantas shown in FIG. 14C. In this case, the shift amount may be determinedby generating a random number.

The modification pattern selection unit 233 notifies the modificationunit 232 of the modification pattern from which a prohibited pattern isexcluded, among modification patterns notified by the random numbergeneration unit 234, in the case where the prohibited pattern isspecified as additional information. On the other hand, in the casewhere a permitted pattern is specified as additional pattern, themodification pattern selection unit 233 notifies the modification unit232 of only permitted patterns among modification patterns notified bythe random number generation unit 234. Also, in the case where neither apermitted pattern nor a prohibited pattern is specified as additionalinformation, all the modification patterns notified by the random numbergeneration unit 234 are notified to the modification unit 232.

The modification unit 232 generates a superimposition pattern bymodifying the decoded representative pattern according to themodification pattern notified by the modification pattern selection unit233.

FIG. 15 is a block diagram showing the second structural example of thefilm grain preparation unit 230.

The film grain preparation unit 230 includes a variable length decodingunit 231, a modification unit 232, and a modification pattern holdingunit 236. Note that the variable length decoding unit 231 and themodification unit 232 are the same as the ones shown in FIG. 12.

The modification pattern holding unit 236 holds the modification patternthat has been previously set. As for this modification pattern, in thecase of the example shown in FIG. 13, the modification pattern and theorder of the modification pattern are held. Also, in the case of theexample shown in FIG. 14, the shift amount and the order of shifting bythe shift amount are held.

The synthesizing unit 240 synthesizes the decoded image data outputtedfrom the decoding unit 210 with the superimposition pattern outputted bythe film grain preparation unit 230 as shown in FIG. 16.

Next, the operation performed by the moving picture decoding apparatusstructured like above will be described. Here will be described the casewhere the film grain preparation unit 230 is the first structuralexample. FIG. 17 is a flow chart showing the operation of the movingpicture decoding apparatus in this case.

Also, the decoding unit 210 decodes the coded stream sent from themoving picture coding apparatus 100 (Step 201). Also, the variablelength decoding unit 231 of the film grain preparation unit 230 performsvariable length decoding on the representative pattern and theadditional information that have been coded by the moving picture codingapparatus 100 and sent (Step 202). Next, the random number generationunit 234 generates a modification pattern by generating a random number(Step 203).

The modification pattern selection unit 233 judges whether or not theprohibited pattern is specified as additional information (Step 204). Inthe case where a prohibited pattern is specified as the judgment, themodification pattern from which the prohibited pattern among themodification patterns notified by the random number generation unit 234is notified to the modification unit 232 (Step 205). On the other hand,in the case where no prohibited pattern is specified, all themodification patterns notified by the random number generation unit 234are notified to the modification unit 232. The modification unit 232generates a superimposition pattern by modifying the decodedrepresentative pattern according to the modification pattern notified bythe modification pattern selection unit 233 (Step 206).

The synthesizing unit 240 synthesizes the decoded image data outputtedby the decoding unit 210 with the superimposition pattern outputted bythe film grain preparation unit 230 (Step 207).

As described up to this point, the moving picture coding apparatus 100codes and sends the film grain components equivalent to at least onemacro block as a representative pattern, separately from the codedstream in which inputted images are coded, and the moving picturedecoding apparatus 200 superimposes a superimposition pattern generatedbased on the representative pattern on the decoded image data obtainedby decoding a coded stream. Therefore, even in the case where inputtedimages are coded in a low bit rate, it is possible to improve thereproducibility by recovering the film grains. Also, since the number ofrepresentative patterns to be sent is few, the amount of additionalinformation is also little. Further, making variations of suchrepresentative patterns using various ways where random numbers are usedmakes it possible to generate film grain components that are visuallynatural.

Note that the random number generation unit 234 may be the structure forobtaining, for example, the picture numbers or time information on thepictures to be decoded from the decoding unit 210, and using them asinitial values (seeds) of a random function. In this case, random numbergeneration patterns become the same because the same initial values(seeds) of a random function are used for the same pictures in eachreproduction. The same superimposition pattern is generated for eachreproduction in this way, and thus it is possible to prevent thesuperimposition patterns superimposed on the decoded image data frombeing changed.

SECOND EMBODIMENT

This embodiment will describe the case where only a part of macroblocks, among the pictures to be coded, are coded with high qualityinstead of coding representative patterns separately.

FIG. 18 is a block diagram showing the structure of the coding unit 140and the film grain coding unit 150 of the moving picture codingapparatus in the second embodiment of the present invention. Note thatthe same parts as the ones in the first embodiment are assigned the samereference numbers, and the descriptions of them will be omitted.

This embodiment differs from the first embodiment in the operationsperformed by (a) the control unit 141 of the coding unit 140, and (b)the selection unit 151 of the film grain coding unit 150 and thevariable length coding unit 152.

The selection unit 151 of the film grain coding unit 150 notifies thevariable length coding unit 152 and the control unit 141 of the codingunit 140, of the position information of the macro block selected likein the first embodiment. The variable length coding unit 152 performsvariable length coding of the position information of the selected macroblock.

The control unit 141 instructs the prediction residual coding unit 112(a) to set the quantization parameter of the macro block specified bythe notified position information lower than the quantization parameterof the other blocks of the picture to be coded (for example, sets thequantization parameter at 0) or (b) to code the macro block again usinga predetermined quantization parameter (for example, the smallestquantization parameter). The prediction residual coding unit 112performs coding processing using the specified quantization parameter,and generates coded data.

In this way, only the macro blocks specified by the position informationin the pictures to be coded are coded with high quality and withoutdamaging film grains.

FIG. 19 is a block diagram showing the structure of the decoding unit250 of the moving picture decoding apparatus in the second embodiment ofthe present invention. FIG. 20 is a block diagram showing the firststructural example of the film grain preparation unit 260. FIG. 21 is ablock diagram showing the second structural example of the film grainpreparation unit 260. Note that the same parts as the ones in the firstembodiment are assigned the same reference numbers and the descriptionsof them will be omitted.

In the embodiment, the prediction residual coded data generated by theprediction residual decoding unit 212 of the decoding unit 250 areinputted also to the film grain preparation unit 260.

The film grain preparation unit 260 includes a film grain patternobtainment unit 261 instead of the variable length decoding unit 231 ofthe first embodiment.

The film grain pattern obtainment unit 261 decodes the positioninformation sent by the moving picture coding apparatus. Also, the filmgrain pattern obtainment unit 261 calculates the prediction residualcoded data of the macro block specified by the position information andthe differential between the macro block and another macro blockreferred to by the macro block, and outputs the differential as arepresentative pattern to the modification unit 232. In other words,since the macro block specified by the position information has codedwith high quality, calculating the differential between the macro blockand another macro block referred to by the macro block makes it possibleto extract the same components as in the case of the representativepattern in the first embodiment.

As described up to this point, only a part of the macro blocks in thepictures to be coded are coded with high quality, and extracting thesame components as the case of the representative pattern in the firstembodiment makes it possible to improve the reproducibility byrecovering film grains, even in the case where the inputted images arecoded in a low bit rate like in the first embodiment. Also, there is amerit that decoding that enables recovering film grains can be realizedin a single coded stream, with this embodiment.

THIRD EMBODIMENT

The above-described first and second embodiments have described filmgrain components, but this embodiment will describe the case ofprocessing high-definition components included in fine images and thelike having parts where a lot of high-frequency components are included.Since such film grain components are high-definition components, thesame method is applicable for the high-frequency components except thefilm grain components.

FIG. 22 is a block diagram showing the overall configuration of a movingpicture coding apparatus and a moving picture decoding apparatus thatuse the moving picture coding method and the moving picture decodingmethod in a third embodiment of the present invention.

The moving picture coding apparatus 300 is for coding high-definitioncomponents separately from the main images, and includes a coding unit310 and a high-definition coding unit 330 as shown in FIG. 22. On theother hand, the moving picture decoding apparatus 400 is forsuperimposing such high-definition components on the decoded images, andincludes a decoding unit 410, a high-definition component decoding unit420, and a high-definition component superimposition unit 430 as shownin FIG. 22.

The coding unit 310 of the moving picture coding apparatus 300 and thehigh-definition component coding unit 330 have the same structures as inthe first embodiment respectively. This embodiment differs in thatsuperimposing position information on the representative pattern and thegain information are specified as additional information, in the blockselection unit 1324 of the selection unit 132 of the high-definitioncoding unit 330, the representative pattern being the high-definitioncomponents to be outputted. This gain information is for specifying, forexample, by what times the pixel values of the representative pattern ismultiplexed in superimposition.

The decoding unit 410 of the moving picture decoding apparatus 400 hasthe same structure as the decoding unit 210 in the first embodiment.Also, the high-definition component decoding unit 420 and thehigh-definition component superimposition unit 430 of the moving picturedecoding apparatus 400 perform the same operations as the ones that thefilm grain preparation unit 230 and the synthesizing unit 240 do in thefirst embodiment. This embodiment differs in that the values obtained bymultiplexing the pixel values of the representative pattern by severaltimes based on the gain information specified like described above, andthe values are superimposed on the superimposing position specified likedescribed above of the decoded image data decoded by the decoding unit210.

As described up to this point, since the representative pattern that isthe high-definition components is superimposed on only a predeterminedposition by specifying the position, it is possible to improve thereproducibility by recovering the high-definition components included infine images and the like having parts where a lot of high-frequencycomponents are included, even in the case where inputted images arecoded in a low bit rate.

FOURTH EMBODIMENT

In the above-described third embodiment, position information forsuperimposing a representative pattern is sent by the moving picturecoding apparatus, but in this embodiment, the case where the movingpicture decoding apparatus determines a superimposing position will bedescribed.

FIG. 23 is a block diagram showing the structure of the moving picturedecoding apparatus in the fourth embodiment of the present invention.

The moving picture decoding apparatus of this embodiment includes asuperimposing position determination unit 520 in addition to thestructure of the third embodiment.

The superimposing position determination unit 520 determines thesuperimposition position for superimposing the representative pattern onthe decoded image data decoded by the decoding unit 210. Thesuperimposing position determination unit 520 judges that there arehigh-frequency components in the case where a single non-0 coefficienttransformed using, for example, DCT (discrete cosine transform) isincluded in a predetermined range of a macro block, and determines thatthe representative pattern is superimposed on the macro block. Note thatthe judgment that there are high-frequency components is not always madeaccording to the above condition, it may be judged that there arehigh-frequency components in the case where, for example, “n” numbers ofnon-0 coefficients are included in a predetermined range of macroblocks. Also, it may be judged that high-frequency components areincluded in macro blocks whose absolute value total of the respectivecoefficients is a predetermined value or more.

As described up to this point, since the moving picture decodingapparatus side superimposes a representative pattern that is thehigh-definition components on only the predetermined position byspecifying the superimposing position, it is possible to improve thereproducibility by recovering the high-definition components included infine images and the like having parts where a lot of high-frequencycomponents are included, even in the case where inputted images arecoded in a low bit rate. Also, there is no need to send suchsuperimposition information from the moving picture coding apparatus tothe moving picture decoding apparatus.

FIFTH EMBODIMENT

Further, recording, on a recording medium such as a flexible disc, aprogram for realizing the moving picture coding method and the movingpicture decoding method shown in the above-described first embodimentmakes it possible to cause an independent computer to execute theprocessing shown in the first embodiment easily.

FIG. 24 is an illustration in the case where the moving picture codingmethod and the moving picture decoding method in the above-describedfirst embodiment that are stored in a flexible disc are executed by acomputer system.

FIG. 24B is an illustration of the front view and the side view of thecase of the flexible disc and the front view of the flexible disc body,and FIG. 24A is an illustration showing an example of a physical formatof a flexible disc that is a recording medium.

A flexible disc (FD) is contained in a case F, a plurality of tracks(Tr) are formed concentrically on the surface of the disc from theperiphery into the inner radius of the disc, and each track is segmentedinto 16 sectors (Se) in the angular direction. Therefore, in the case ofthe flexible disc storing the above-described program, the movingpicture coding method as the program is recorded in an area allocatedfor it on the flexible disc (FD). Also, FIG. 24C shows the structure forrecording and reproducing the program on the flexible disc (FD). In thecase where the program is recorded on the flexible disc (FD), thecomputer system (Cs) writes the moving picture coding method or themoving picture decoding method as the program by the computer system Cs.Also, in the case where the moving picture coding method described aboveis constructed in the computer system by the program on the flexibledisc, the program is read out from the flexible disc through a flexibledisc drive and transferred to the computer system.

Note that the above description has described a flexible disc asrecording medium, but such description may describe an optical disc.Also, such a recording medium is not limited to this, and suchdescription may also describe an IC card, a ROM cassette and the like aslong as it is a medium for recording the program.

Further, an application example of the moving picture coding method andthe moving picture decoding method shown in the above-describedembodiments and the system where they are used will be described here.

FIG. 25 is an illustration showing the overall configuration of thecontent supply system ex100 that realizes a content distributionservice. The area for providing communication service is segmented intocells of desired sizes, and cell sites ex107 to ex110 of fixed wirelessstations are placed in the cells respectively. This content supplysystem ex100 is connected to each apparatus such as a computer ex111, aPersonal Digital Assistant (PDA) ex112, a camera ex113, a mobile phoneex114 and a mobile phone with a camera ex115 via, for example, acombination of the Internet ex101, an Internet service provider ex102, atelephone network ex104 and cell sites ex107 to ex110.

However, the content supply system ex100 is not limited to theconfiguration as shown in FIG. 25, and may be connected to a combinationof any of them. Also, each apparatus can be connected directly to thetelephone network ex104, not through the cell sites as fixed radiostations ex107 to ex110.

The camera ex113 is an apparatus capable of shooting video (movingpictures) such as a digital video camera. The mobile phone may be amobile phone of a Personal Digital Communications (PDC) system, a CodeDivision Multiple Access (CDMA) system, a Wideband-Code DivisionMultiple Access (W-CDMA) system or a Global System for MobileCommunications (GSM) system, a Personal Handy-phone system (PHS) or thelike.

A streaming server ex103 is connected to the camera ex113 via the cellsite ex109 and the telephone network ex104, which enables livedistribution or the like using the camera ex113 based on the coded datatransmitted from the user. Either the camera ex113 or the server fortransmitting the data can code the shot data. Also, the moving picturedata shot by a camera ex116 can be transmitted to the streaming serverex103 via the computer ex111. The camera ex116 is an apparatus such as adigital camera that is capable of shooting still and moving pictures. Inthis case, either the camera ex116 or the computer ex111 may performcoding of the moving picture data. Also, in the computer ex111 or in anLSI ex117 included in the camera ex116, the coding processing isperformed. Note that software for coding and decoding moving picturesmay be integrated into any type of recording media (such as a CD-ROM, aflexible disc, a hard disc and the like) that is a lo recording mediumwhich is readable by the computer ex115 or the like. Furthermore, amobile phone with a camera ex115 may transmit the moving picture data.This moving picture data is the data coded by the LSI included in themobile phone ex115.

The content supply system ex100 codes contents (such as a music livevideo) shot by users using the camera ex113, the camera ex116 or thelike in the same manner as the above-described embodiments and transmitsthem to the streaming server ex103. Meanwhile, the streaming serverex103 makes stream distribution of the contents data to the clients upontheir request. The clients include the computer ex111, the PDA ex112,the camera ex113, the mobile phone ex114 and so on that are capable ofdecoding the above-described coded data. In this way, the content supplysystem ex100 enables the clients to receive and reproduce the codeddata, and further to receive, decode and reproduce the data in real timeso as to realize personal broadcasting.

The moving picture coding apparatus or the moving picture decodingapparatus shown in the above-described embodiments may be used in codingand decoding performed by the respective apparatuses that constitutethis system.

A mobile phone will be described as an example.

FIG. 26 is a diagram showing the mobile phone ex115 using the movingpicture coding method and the moving picture decoding method describedin the earlier embodiments. The mobile phone ex115 has an antenna ex201for communicating with the cell site ex110 via radio waves, a cameraunit ex203 such as a CCD camera that is capable of shooting moving andstill pictures, a display unit ex202 such as a liquid crystal displaythat displays the data obtained by decoding moving pictures and the likereceived via the antenna ex201, a body unit including a set of operationkeys ex204, a sound output unit ex208 such as a speaker for outputtingsound, a sound input unit 205 such as a microphone for inputting sound,a recording medium ex207 for storing coded or decoded data such as dataof moving or still pictures, data of received e-mail and data of movingor still pictures, and a slot unit ex206 operable to attach therecording medium ex207 to the mobile phone ex115. The storage mediumex207 is equipped with a flash memory element, a kind of ElectricallyErasable and Programmable Read Only Memory (EEPROM) that is anelectrically erasable and rewritable nonvolatile memory such as an SDcard in a plastic case.

Further, the mobile phone ex115 will be described with reference to FIG.27. In the mobile phone ex115, a main control unit ex311, which isoperable to perform centralized control on each unit of the bodyincluding the display unit ex202 and operation keys ex204, is connectedto a power supply circuit unit ex310, an operation input control unitex304, a picture coding unit ex312, a camera interface unit ex303, aLiquid Crystal Display (LCD) control unit ex302, a picture decoding unitex309, a multiplexing/demultiplexing unit ex308, a recording andreproducing unit ex307, a modem circuit unit ex306 and a soundprocessing unit ex305 via a synchronous bus ex313.

When a call-end key or a power key is turned ON by a user's operation,the power supply circuit unit ex310 supplies respective units with powerfrom a battery pack so as to activate the digital mobile phone with acamera ex115 for making it into a ready state.

In the cell phone ex115, the sound processing unit ex305 converts thesound signals received by the sound input unit ex205 in conversationmode into digital sound data under the control of the main control unitex311 including a CPU, a ROM and a RAM. The modem circuit unit ex306performs spread spectrum processing of the digital sound data, and thetransmission circuit unit ex301 performs digital-to-analog conversionand frequency transform processing of the data so as to transmit it viathe antenna ex201. Also, in the mobile phone ex115, the transmissioncircuit unit ex301 amplifies the data received by the antenna ex201 inconversation mode and performs frequency transform processing andanalog-to-digital conversion processing on the data. The modem circuitunit ex306 performs inverse spread spectrum processing on the data, thesound processing unit ex305 converts it into analog sound data, and thesound output unit ex208 outputs it.

Furthermore, when transmitting e-mail in data communication mode, thetext data of the e-mail inputted by operating the operation keys ex204on the body unit is sent out to the main control unit ex311 via theoperation input control unit ex304. In the main control unit ex311,after the modem circuit unit ex306 performs spread spectrum processingon the text data and the transmission circuit unit ex301 performsdigital-to-analog conversion processing and frequency transformprocessing on it, the data is transmitted to the cell site ex301 via theantenna ex201.

When picture data is transmitted in data communication mode, the picturedata shot by the camera unit ex203 is supplied to the picture codingunit ex312 via the camera interface unit ex303. When the picture data isnot transmitted, it is also possible to display the picture data shot bythe camera unit ex203 directly on the display unit 202 via the camerainterface unit ex303 and the LCD control unit ex302.

The picture coding unit ex312, which includes the moving picture codingapparatus that have been described in the present invention, performscompression coding on the picture data supplied from the camera unitex203 using the coding method used for the moving picture codingapparatus that have been shown in this application of the presentinvention so as to convert it into coded picture data, and sends it outto the multiplexing/demultiplexing unit ex308.

The multiplexing/demultiplexing unit ex308 multiplexes the coded picturedata supplied from the picture coding unit ex312 and the sound datasupplied from the sound processing unit ex305 using a predeterminedmethod, the modem circuit unit ex306 performs spread spectrum processingon the multiplexed data obtained as a result of the multiplexing, andthe transmission circuit unit ex301 performs digital-to-analogconversion and frequency transform processing on the data fortransmitting via the antenna ex201.

As for receiving data of a moving picture file which is linked to a Webpage or the like in data communication mode, the modem circuit unitex306 performs spread spectrum processing of the data received from thecell site ex110 via the antenna ex201, and sends out the multiplexeddata obtained as a result of the processing to themultiplexing/demultiplexing unit ex308.

In order to decode the multiplexed data received via the antenna ex201,the multiplexing/demultiplexing unit ex308 separates the multiplexeddata into a bit stream of picture data and a bit stream of sound data,and supplies the current coded picture data to the picture decoding unitex309 and the current sound data to the sound processing unit ex305respectively via the synchronous bus ex313.

Next, the picture decoding unit ex309, which includes the moving picturedecoding apparatus that have been described in this application of thepresent invention, decodes the bit stream of picture data using thedecoding method corresponding to the coding method as shown in theabove-described embodiments to generate reproduced moving picture data,and supplies this data to the display unit ex202 via the LCD controlunit ex302, and thus, for instance, the moving picture data included ina moving picture file linked to a Web page is displayed. At the sametime, the sound processing unit ex305 converts the sound data intoanalog sound data, and supplies this data to the sound output unitex208, and thus, for instance, sound data included in a moving picturefile linked to a Web page is reproduced.

Note that the present invention is not limited to the above-describedsystem, and at least either the moving picture coding apparatus or themoving picture decoding apparatus in the above-described embodiments canbe incorporated into a system for digital broadcasting as shown in FIG.28. Such ground-based or satellite digital broadcasting has been in thenews lately. More specifically, a bit stream of video information istransmitted from a broadcast station ex409 to a communication or abroadcast satellite ex410 via radio waves. Upon receipt of it, thebroadcast satellite ex410 transmits radio waves for broadcasting, ahome-use antenna ex406 with a satellite broadcast reception functionreceives the radio waves, and a television (receiver) ex401, a set topbox (STB) ex407 or the like decodes and reproduces the bit stream. Themoving picture decoding apparatus described in the above embodiments canbe implemented in the reproduction apparatus ex403 for reading out anddecoding the bit stream recorded on a storage medium ex402 that is arecording medium such as a CD and a DVD. In this case, the reproducedvideo signals are displayed on a monitor ex404. It is also conceived toimplement the moving picture decoding apparatus in the set top box ex407connected to a cable ex405 for a cable television or the antenna ex406for satellite and/or ground-based broadcasting so as to reproduce themon a monitor ex408 of the television. The moving picture decodingapparatus may be incorporated into the television, in stead of in theset top box. Otherwise, a car ex412 having an antenna ex411 can receivesignals from the satellite ex410, the cell site ex107 or the like forreproducing moving pictures on a display apparatus such as a carnavigation system ex413.

Furthermore, the moving picture coding apparatus described in the aboveembodiments can code picture signals for recording them on a recordingmedium. As a concrete example, there is a recorder ex420 such as a DVDrecorder for recording picture signals on a DVD disc ex421 and a discrecorder for recording them on a hard disc. They can also be recorded onan SD card ex422. If the recorder ex420 includes the moving picturedecoding apparatus described in the above embodiment, the picturesignals recorded on the DVD disc ex421 or the SD card ex422 can bereproduced for display on the monitor ex408.

Note that a conceivable configuration of the car navigation system ex413is the configuration obtained by eliminating the camera unit ex203, thecamera interface unit ex303 and the picture coding unit ex312 from theexisting components in FIG. 27. The same goes for the computer ex111,the television (receiver) ex401 and the like.

In addition, three types of implementation can be conceived for aterminal such as the above-described cell phone ex114, asending/receiving terminal implemented with both an encoder and adecoder, a sending terminal implemented with an encoder only, and areceiving terminal implemented with a decoder only.

In this way, it is possible to use the moving picture coding apparatusor the moving picture decoding apparatus in the above-describedembodiments in any of the above-described apparatuses and systems, andby using this method, the effects described in the above embodiments canbe obtained.

The present invention is not limited to the above-described embodiments,and it will be obvious that the embodiments of the invention may bevaried in many ways.

Note that the respective functional blocks of the respective embodimentsare typically realized as an LSI that is a large scale integrationcircuit. Each functional blocks may be made into a single chip, or apart of or all of the functional blocks may be made into a single chipall together (for example, functional blocks except a memory may be madeinto a single chip).

The integrated circuit is called LSI here, but it may be called IC,system LSI, super LSI, or ultra LSI, depending on the integrationdegree.

Also, the method of making them into an integrated circuit is notlimited to the method of making them into an LSI, it may be realized byan exclusive circuit or a multi-purpose processor. Also, it is possibleto use (a) a reconfigurable processor where the connection or thesetting of circuit cells can be reconfigured or (b) a programmable FPGA(Field Programmable Gate Array), after making them into an LSI.

Further, in the case where technique of making them into an integratedcircuit instead of making them into an LSI appears when thesemiconductor technique is further developed or any derivative techniqueappears, in due course, functional blocks may be made into an integratedcircuit using such new technique. Application of bio technique islikely.

Also, among respective functional blocks a storage unit (a picturememory) in which the picture data to be coded or decoded is stored maybe configured separately instead of being included in a single chip.

INDUSTRIAL APPLICABILITY

As described up to this point, the moving picture coding method and amoving picture decoding method in the present invention are applicableas methods for (a) generating a coded stream by coding picturesconstituting a moving picture using, for example, a mobile phone, a DVDapparatus, a personal computer and the like, and (b) decoding thegenerated coded stream.

1. A moving picture coding method for coding each picture constituting amoving picture on a block-by-block basis, comprising: a first codingstep of coding a current picture to be coded; an extraction step ofextracting high-definition components from the current picture to becoded; and a selection step of selecting high-definition componentsequivalent to at least one block, from the high-definition componentsextracted in said extraction step.
 2. The moving picture coding methodaccording to claim 1, further comprising a second coding step for codingthe high-definition components equivalent to the at least one block, thehigh-definition components being selected in said selection step.
 3. Themoving picture coding method according to claim 2, wherein, in saidsecond coding step, a value of each high-definition component is coded.4. The moving picture coding method according to claim 2, wherein, insaid second coding step, each high-definition component is transformedinto coefficients indicating spatial frequency components, and thecoefficients are coded.
 5. The moving picture coding method according toclaim 1, wherein, in said first coding step, the at least one blockwhich is selected in said selection step is coded again by any of (a)setting a quantization parameter of the at least one block which isselected in said selection step, the quantization parameter being lowerthan quantization parameters of other blocks of the current picture tobe coded and (b) using a predetermined quantization parameter.
 6. Themoving picture coding method according to claim 1, wherein, in saidextraction step, the high-definition components are extracted bycalculating a differential between the current picture to be coded and adecoded picture obtained by decoding the current picture after beingcoded.
 7. The moving picture coding method according to claim 1,wherein, in said selection step, the high-definition componentsequivalent to the at least one block are selected on thepicture-by-picture basis, the picture being the current picture to becoded.
 8. The moving picture coding method according to claim 1,wherein, in said selection step, each variance of the high-definitioncomponents is calculated on the block-by-block basis, thehigh-definition components being extracted in said extraction step, andthe high-definition components equivalent to the at least one block areselected based on each calculated variance.
 9. The moving picture codingmethod according to claim 8, wherein, in said selection step, thehigh-definition components of a block are selected, the block having avariance which is indicated by a frequency peak in a histogram of thecalculated variances.
 10. The moving picture coding method according toclaim 8, wherein, in said selection step, the high-definition componentsof a block are selected, the block having a calculated variance which isequal to a predetermined variance.
 11. The moving picture coding methodaccording to claim 1, wherein, in said selection step, the followingeach median is calculated: the median being of pixels of a correspondingpositions of blocks which include the high-definition componentsextracted in said extraction step, and the calculated median is selectedas the high-definition components for one block.
 12. The moving picturecoding method according to claim 1, wherein, in said selection step,additional information is assigned to the selected high-definitioncomponents.
 13. The moving picture coding method according to claim 12,wherein, in said selection step, any of a permitted pattern insuperimposition and a prohibited pattern in superimposition of theselected high-definition components is specified as the additionalinformation on the picture-by-picture basis.
 14. The moving picturecoding method according to claim 12, wherein, in said selection step,any of a permitted pattern in superimposition and a prohibited patternin superimposition of the selected high-definition components isspecified as the additional information on the high-definition componentbasis.
 15. The moving picture coding method according to claim 12,wherein, in said selection step, a superimposing position insuperimposition of the selected high-definition components are specifiedas the additional information.
 16. The moving picture coding methodaccording to claim 12, wherein, in said selection step, a gain insuperimposition of the selected high-definition components is specifiedas the additional information.
 17. The moving picture decoding methodfor decoding a coded stream obtained by coding each constituent pictureon a block-by-block basis, comprising: a first decoding step ofgenerating decoded picture data by decoding the coded stream; anobtainment step of obtaining high-definition components equivalent tothe at least one block; and a superimposition step of superimposing thehigh-definition components equivalent to the at least one block on thedecoded picture data, the high-definition components being obtained insaid obtainment step.
 18. The moving picture decoding method accordingto claim 17, further comprising a modification step of modifying thehigh-definition components equivalent to the at least one block, thehigh-definition components being obtained in said obtainment step,wherein, in said superimposition step, the high-definition componentsmodified in said modification step is superimposed on the decodedpicture data.
 19. The moving picture decoding method according to claim18, further comprising a pattern preparation step of preparing amodification pattern indicating what modification of the high-definitioncomponents equivalent to the at least one block is made, thehigh-definition components being obtained in said obtainment step,wherein, in said modification step, the high-definition components aremodified based on the modification pattern.
 20. The moving picturedecoding method according to claim 19, wherein, in said patternpreparation step, the at least one of the following modificationpatterns is prepared: rotation; reversal; level changing; and positionshifting.
 21. The moving picture decoding method according to claim 19,wherein, in said pattern preparation step, the modification pattern isprepared at random based on a random function.
 22. The moving picturedecoding method according to claim 21, wherein, in said patternpreparation step, the modification pattern is prepared on the premisethat an initial value of the random function is constant.
 23. The movingpicture decoding method according to claim 21, wherein, in said patternpreparation step, an initial value of the random function is obtainedfrom the coded stream.
 24. The moving picture decoding method accordingto claim 21, wherein, in said obtainment step, initial information isfurther obtained, and in said pattern preparation step, the modificationpattern is prepared, setting the initial information as an initial valueof the random function, the initial information being obtained in saidobtainment step.
 25. The moving picture decoding method according toclaim 19, wherein, in said pattern preparation step, a predeterminedmodification pattern and an order of the modification pattern arepreviously held.
 26. The moving picture decoding method according toclaim 19, wherein, in said obtainment step, a prohibited pattern isfurther obtained, and in said modification step, the high-definitioncomponents are modified based on a modification pattern excluding theprohibited pattern from the modification patterns.
 27. The movingpicture decoding method according to claim 17, further comprising asuperimposing position obtainment step of obtaining a superimposingposition, wherein, in said superimposition step, the high-definitioncomponents are superimposed on the superimposing position of the decodedpicture data, the superimposing position being obtained in saidsuperimposing position obtainment step.
 28. The moving picture decodingmethod according to claim 27, wherein, in said superimposing positionobtainment step, the superimposing position being obtained based on thecoded stream.
 29. A moving picture coding apparatus which codes eachpicture constituting a moving picture on a block-by-block basis,comprising: a first coding unit operable to code a current picture to becoded; an extraction unit operable to extract high-definition componentsfrom the current picture to be coded; and a selection unit operable toselect the high-definition components equivalent to at least one block,from the high-definition components extracted by said extraction unit.30. A moving picture decoding apparatus which decodes a coded streamobtained by coding each constituent picture on a block-by-block basis,comprising: a first decoding unit operable to generate decoded picturedata by decoding the coded stream; an obtainment unit operable to obtainhigh-definition components equivalent to at least one block; and asuperimposition unit operable to superimpose the high-definitioncomponents equivalent to the at least one block on the decoded picturedata, the high-definition components being obtained by said obtainmentunit.
 31. A program for causing a computer to execute coding of eachpicture constituting a moving picture on a block-by-block basis, thecoding comprising: a first coding step of coding a current picture to becoded; an extraction step of extracting high-definition components fromthe current picture to be coded; and a selection step of selecting thehigh-definition components equivalent to at least one block, from thehigh-definition components extracted in said extraction step.
 32. Aprogram for causing a computer to execute decoding of a coded streamobtained by coding each constituent picture on a block-by-block basis,the decoding comprising: a first decoding step of generating decodedpicture data by decoding the coded stream; an obtainment step ofobtaining high-definition components equivalent to the at least oneblock; and a superimposition step of superimposing the high-definitioncomponents equivalent to the at least one block on the decoded picturedata, the high-definition components being obtained in said obtainmentstep.
 33. An integrated circuit for coding each picture constituting amoving picture on a block-by-block basis, comprising: a first codingunit operable to code a current picture to be coded; an extraction unitoperable to extract high-definition components from the current pictureto be coded; and a selection unit operable to select the high-definitioncomponents equivalent to at least one block, from the high-definitioncomponents extracted by said extraction unit.
 34. An integrated circuitfor decoding a coded stream obtained by coding each constituent pictureon a block-by-block basis, comprising: a first decoding unit operable togenerate decoded picture data by decoding the coded stream; anobtainment unit operable to obtain high-definition components equivalentto at least one block; and a superimposition unit operable tosuperimpose the high-definition components equivalent to the at leastone block on the decoded picture data, the high-definition componentsbeing obtained by said obtainment unit.
 35. A coded stream includinghigh-definition components equivalent to at least one block, thehigh-definition components to be superimposed on decoded picture dataobtained by decoding a coded stream whose constituent pictures are codedon a block-by-block basis.